📌 Exploratory Data Analysis (EDA)

Monthly Trend of Data-Related Job Postings by Role

# 1. Filter the main df into df_roles
df_roles = df[df["TITLE_NAME"].str.contains(" Data Analysts|Business Intelligence Analysts|Data Analytics Engineers|Data and Reporting Analysts|"
    "Data Governance Analysts|Data Quality Analysts|Data Analytics Analysts|Data Management Analysts|"
    "Data Modelers|Lead Data Analysts|Research Data Analysts|IT Data Analysts|Data Analytics Manager|"
    "Lead Business Intelligence Analysts|Data Science Analysts|Data Analytics Leads|"
    "Business Intelligence Data Analysts|Data Operations Analysts|Health Data Analysts|"
    "Data analytics consultants|Enterprise Data Analysts", case=False, na=False)]

# 1. Make a copy to avoid SettingWithCopyWarning
df_roles = df_roles.copy()

# 2. Convert POSTED to datetime (if not already)
df_roles["POSTED"] = pd.to_datetime(df_roles["POSTED"])

# 3. Create a monthly column as a Timestamp
df_roles["month"] = df_roles["POSTED"].dt.to_period("M").dt.to_timestamp()

# 4. Group by month and role
monthly_counts = df_roles.groupby(["month", "TITLE_NAME"]).size().reset_index(name="count")

fig = px.area(
    monthly_counts,
    x="month",
    y="count",
    color="TITLE_NAME",
    title="Monthly Trend of Data-Related Job Postings by Role (Stacked Area)"
)
fig.show()

In the above Stacked Area Chart, we focused on understanding the monthly trends for various data-related job roles. To do this, we filtered the dataset to include only job titles containing keywords like “Data Analysts,” “Business Intelligence Analysts,” and “Data Engineers,” among others. After filtering, we converted the posting dates into a proper datetime format and created a new column to track monthly trends. We then grouped the data by month and job title to visualize how the demand for these roles evolved over time.

The stacked area chart we generated illustrates that the demand for data-related roles has shown a consistent upward trend, especially after mid-June 2024. One of the most noticeable patterns is the significant share held by roles such as Big Data Analysts and Business Intelligence Analysts, which suggests that organizations are increasingly seeking professionals who can handle large-scale data analysis and provide actionable insights.

Interestingly, we observed that the demand for Data Analysts and Data Analytics Engineers also increased steadily, indicating a growing need for professionals who can not only interpret data but also manage the infrastructure required for analysis. This aligns with the broader industry trend of integrating advanced analytics into decision-making processes.

We also found that more specialized roles like Clinical Data Analysts and Customer Data Analysts are gaining traction, although they represent a smaller share of the total postings. This could imply that industries such as healthcare and customer analytics are gradually expanding their analytics teams.

Remote vs. On-Site Jobs (Data Roles)

fig = px.pie(df_roles, names="REMOTE_TYPE_NAME", title="Remote vs. On-Site Jobs")
fig.show()

We created a Pie Chart to explor the distribution of remote, on-site, and hybrid roles for data-related jobs. The chart shows that 73.8% of postings do not specify a preference, suggesting either data gaps or employer flexibility. Among specified roles, 20.4% are remote, indicating a strong demand for remote work. Hybrid roles account for 4%, while fully on-site roles are only 1.79%.

These findings suggest a clear shift towards remote and flexible work arrangements in the data field. Highlighting remote work skills could be advantageous for job seekers in this area.

Data-Related Job Postings Across the USA

# 🔹 Step 2: Aggregate Job Counts by State
job_counts = df_roles["STATE_NAME"].value_counts().reset_index()
job_counts.columns = ["state", "job_postings"]

# 🔹 Step 3: Convert Full State Names to Abbreviations
job_counts["state"] = job_counts["state"].apply(lambda x: us.states.lookup(x).abbr if us.states.lookup(x) else x)

# 🔹 Step 4: Create Choropleth Map
fig = px.choropleth(
    job_counts,
    locations="state",
    locationmode="USA-states",  # Correct location mode for US states
    color="job_postings",  # Color by job count
    hover_name="state",
    color_continuous_scale="Viridis",  # High-contrast color scheme
    title="Data-Related Job Postings Across the USA",
    scope="usa"  # Focus on the USA
)

fig.show()

We created a Choropleth Map to understand the geographical distribution of data-related job postings across the United States. To do this, we aggregated the job counts by state and converted full state names to abbreviations to ensure consistency in the map. We then used a choropleth map to visualize how job postings are spread across different states.

From the map, we can see that states like California, Texas, and Florida have the highest concentrations of data-related job postings, indicated by the brighter yellow shades. This pattern suggests that these states are major hubs for data-related roles, likely due to their large tech industries and the presence of major corporations.

In contrast, several states in the Midwest and some in the Mountain region show darker shades, indicating fewer job opportunities in data-related fields. This could imply either a lower demand for such roles or a smaller tech industry presence in those areas.

Industries Having Data-Related Roles

# Get industries associated with these roles
industry_text = " ".join(df_roles["NAICS_2022_6_NAME"].dropna())

# Generate word cloud
wordcloud = WordCloud(
    width=800, height=400, background_color="white", 
    colormap="viridis", max_words=100
).generate(industry_text)

# Display the word cloud
plt.figure(figsize=(12, 6))
plt.imshow(wordcloud, interpolation="bilinear")
plt.axis("off")
plt.title("Industries Having Data-Related Roles", fontsize=14)
plt.show()

With this WordCloud, we aimed to identify the industries that most frequently offer data-related roles. We extracted industry names from the dataset, combined them into a single text string, and then generated a word cloud to visualize which industries appeared most often.

From the word cloud, we can see that “Unclassified Industry,” “Programming Services,” “Consulting Services,” and “Insurance Carriers” are some of the most prominent terms. The large size of these words suggests that these industries have a significant number of data-related job postings. This pattern highlights the growing importance of data roles in both tech-heavy sectors like programming and consulting as well as in more traditional fields such as insurance.

Other notable industries in the word cloud include “Computer Systems Design,” “Employment Placement Agencies,” and “Direct Health Services.” Their visibility suggests a broad demand for data professionals across diverse sectors, emphasizing that data analytics skills are valuable beyond just the tech industry.

This indicate that the demand for data roles is not limited to a few industries but is spread across both tech-focused and traditional sectors. This reinforces the idea that data skills are becoming essential across the board.

📌 Key Findings

Industry Demand:

Programming Services, Consulting Services, and Insurance emerged as leading industries for data-related roles, indicating a widespread need for data skills beyond traditional tech sectors.
The prominence of Unclassified Industry suggests potential gaps in data classification or a diverse range of roles that do not fit into conventional categories.

Geographical Distribution:

California, Texas, and Florida were identified as major hubs for data-related jobs, while several Midwestern and Mountain states showed fewer opportunities.
This pattern suggests that professionals may find more job prospects by focusing on these high-demand states.

Work Arrangement Preferences:

A significant share of job postings preferred remote and hybrid roles, with over 20% specifically offering remote options.
The limited proportion of fully on-site roles reflects a broader shift towards flexible work models in the data industry.

Role-Specific Trends:

There is a clear upward trend in demand for roles such as Big Data Analysts and Business Intelligence Analysts, highlighting the growing importance of both technical and analytical skills.
Specialized roles like Clinical Data Analysts and Customer Data Analysts are also gaining traction, indicating expanding opportunities in niche areas.

📌 Conclsuion

Our analysis revealed that the demand for data-related skills is both substantial and diverse, spanning multiple industries and regions across the United States. Key sectors such as tech, consulting, and insurance show the most significant opportunities, while states like California, Texas, and Florida lead in job postings.

The strong preference for remote and hybrid roles highlights the importance of flexibility in the current job market. Meanwhile, the consistent rise in demand for specialized data roles suggests a promising outlook for professionals equipped with both analytical and domain-specific skills.

Overall, these findings suggest that focusing on high-demand industries, enhancing remote work capabilities, and acquiring specialized skills can significantly boost job prospects in the data field.